Auditory perceptual systems

ABSTRACT

A new hearing profile assessment plays each of a plurality of frequency tones multiple times at a plurality of loudness levels. For each frequency, the patient indicates which tones that they can hear, establishing minimum-amplitude thresholds for each frequency. The assessment adapts loudness levels until the patient indicates consistent minimum-amplitude thresholds for each frequency. Also, a new system for training the hearing of a subject is disclosed. The system instructs the subject to vocalize various sound items and remember the sounds that they vocalize. The system plays either recorded samples of the subject&#39;s own voice, or comparison samples of a synthesized voice or other&#39;s vocalizations of the same sound items. The system prompts the subject to compare the played samples with the sounds they remembered vocalizing. The system uses the feedback to adjust sound processing parameters for a hearing aid, cochlear implant, sound output devices, or a training program.

CROSS REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit of U.S. Provisional Application No.61/782,571, filed Mar. 14, 2013, for a “Novel Strategy to AccelerateRecovery of Speech Understanding in Individuals with Cochlear Implantsor Hearing Aids,” which is herein incorporated by reference for allpurposes.

FIELD OF THE INVENTION

This invention relates in general to hearing health and in particular,to auditory perceptual training to correct hearing loss.

BACKGROUND OF THE INVENTION

Hearing Loss.

Hearing loss effects nearly 40 million Americans, but many individualssuffering from hearing loss or other hearing-related disorders do notseek clinical help because of stigma, financial constraints or otherobstacles, and many people are unaware of their hearing impairment.

Even people fitted with hearing aids show limited success because thedevices do not account for the detrimental neural changes induced byhearing loss that prevent the user from maximally exploiting low-levelspeech cues, such as subtle changes in frequency and timing.

Impaired hearing desensitizes the cortex to hearing-loss frequencies anddegrades its ability to distinguish auditory input. These cognitivedeficits disrupt higher-level auditory processing like phonemediscrimination, ultimately impairing the hearing-impaired individual'sability to engage in and remember conversations.

Hearing aids do not correct this pathological neural function becausethe cortex must be trained to properly discriminate and utilize thefrequencies amplified by the aid along with other auditory informationlost from degraded input.

Auditory perceptual training can reverse the pathological plasticityassociated with hearing loss by training the individual to make thenecessary temporal, intensity and frequency discriminations required forhealthy hearing abilities. As these discriminations are fundamental tophoneme processing, such training can improve speech perception. Infact, phoneme discrimination training has been shown to improve speechperception in hearing-impaired individuals beyond the benefit of ahearing-aid alone. Although previous art has shown that auditoryperceptual training can improve speech perception in hearing-impairedindividuals, such training has not been tailored to individuals'personal hearing needs.

There is a need for individually-tailored auditory perceptual training.There is a concomitant need for practical automated systems and methodsfor tailoring auditory perceptual training to an individual.

Audiometry.

There are many methods of audiometry for diagnosing hearing loss. Butcurrent clinical audiometric methods are not able to be implemented onpersonal computers operable by a layperson; i.e., the sufferer ofhearing loss. Current clinical audiometric devices require a trainedaudiologist to operate and interpret the test results in the form of anaudiogram. Worse, apparently similar audiograms may permit multipleinterpretations because of technical variations in audiometric methodsand test subject idiosyncrasies. These and other complications must beovercome to make audiometric methods simple enough for the sufferer touse themselves, and accessible enough to be used on their own device(such as a personal computer via the internet).

There are additional obstacles to developing a layperson-operableaudiometric device. The sound output by sound cards and headphonesvaries from one device to another. Ensuring accurate sound pressurelevels across the range of tested frequencies requires calibration ofthe sound output to compensate for distinct frequency responses of thecomputer's sound card and the headphones used during the test. Also,sine tones commonly presented in audiometry can produce harmonicdistortion when presented at intense levels. Additionally, personalcomputers produce noise that may interfere with the user's performance.

Also, current audiometric methods focus only on quantifying hearingthresholds. We are aware of no prior art audiometers that informperceptual training software as to what kinds of sound discriminationswould be most useful for the user.

SUMMARY

A new audiometric assessment method is provided. In one embodiment, themethod involves presenting a game that plays a plurality of tonesspanning a plurality of frequencies, with each tone played at aplurality of loudness levels distributed across discrete decibelincrements. A player indicates which of the tones, if any, they canhear. For each frequency, the assessment adapts the loudness levels ofthe tones and replays the tones until the player identifies a consistentminimum-amplitude threshold. The player only compares tones of the samefrequency, and is assessed for a new frequency once a consistent hearingthreshold has been established.

The game challenges the user to indicate as rapidly as possible whetherthey can hear the tones. The game provides sensory feedback on theaccuracy and speed of the user's response.

In an exemplary embodiment, the game displays a plurality of selectablevisual objects, each associated with one of the tones. The player ischallenged to hover a pointer over the objects to play the tones and toselect the objects if they are able to hear the tones.

Advantageously, the audiometer quickly, accurately and consistentlyderives the prominent regions of hearing impairment.

A new audio perception training method and system is also provided. Thesystem instructs the subject to vocalize sound items and remember thesound items that they vocalize. In one embodiment, the system recordssamples of the subject's voice and plays the recorded samples back. Inanother embodiment, the system plays samples of synthesized speech orother persons' voices. The system then prompts the subject to comparethe played sample with the sound they remembered vocalizing and providefeedback on the comparison. The system then adjusts sound processingparameters on the basis of the subject's feedback. In one embodiment,the sound processing parameters calibrate a hearing aid or cochlearimplant worn by the patient. In another embodiment, the sound processingparameters calibrate the output of a sound card, speaker or other soundoutput device.

The system repeatedly procures sound item samples from the subject andfeedback from the subject comparing played samples with the sound orsounds they remembered vocalizing. The system progressively adjusts thesound processing parameters until the subject indicates that they aresatisfied that the played samples sound similar to what they rememberedvocalizing, or a threshold number of adjustments have been made, or anevaluation of the subject's feedback indicates a marginal benefit tofurther adjustments.

In another embodiment, an audiometric or cognitive training programselects optimal stimuli—such as stimuli using frequencies to which thesubject is least sensitive—to use in subsequent games that are afunction of the results obtained from the assessment. Such training,coupled with a hearing aid or cochlear implant, focuses on regions ofnewly-acquired hearing. The cortical regions corresponding to one'shearing loss are largely inactive and poorly tuned despite amplifiedinput from the device. The auditory perceptual training allows corticalresponse to the amplified input to be normalized. To render the trainingeven more effective, the training program uses an audiometer or othermethod of hearing threshold estimation to ensure optimal stimuli areselected for training.

Other features and advantages of the present invention will becomeapparent upon study of the remaining portions of the specification anddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one embodiment of a multi-faceted,web-deliverable, and game-based cognitive training system.

FIG. 2 illustrates a screenshot of one embodiment of an auditoryassessment called “Hearing Profile.”

FIG. 3 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 4 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 5 is a functional block diagram of one embodiment of the auditoryassessment of FIG. 2.

FIG. 6 is a screenshot of one embodiment of an online Tinnitusassessment.

FIG. 7 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 8 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 9 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 10 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 11 is a functional block diagram of one embodiment of aninteroaudioceptive calibration procedure for adjusting the soundprocessing parameters of a hearing aid or cochlear implant.

FIG. 12 is a functional block diagram of an embodiment of aninteroaudioceptive calibration procedure used for improving the efficacyof auditory and cognitive training exercises.

FIG. 13 is a functional block diagram of yet another embodiment of aninteroaudioceptive calibration procedure.

DETAILED DESCRIPTION

Various embodiments of the present invention use a computer system and acomputer network for executing one or more cognitive training computerprograms, where “cognition” refers to the speed, accuracy andreliability of processing of information, including filtering, recall,and manipulation of information, and attention and/or working memory.

A typical computer system (not shown) for use with the present inventionwill contain a computer, having a CPU, memory, hard disk, and variousinput and output devices. A display device, such as a monitor or digitaldisplay, provides visual prompting and feedback to the subject duringexecution of the computer program. Speakers or a pair of headphones orear buds provide auditory prompting and feedback to the subject. Aprinter may be connected to the computer to enable a subject can printout reports associated with the computer program. Input devices such asa keyboard, mouse, trackpad, touch screen, microphone, camera, or othersensor receive input from the subject. A number of different computerplatforms are applicable to the present invention, including but notlimited to embodiments that execute on IBM compatible computers,Macintosh computers, phones, tablets, set top boxes, PDA's, and gamingconsoles.

A computer network (not shown) for use with the present inventioncontains multiple computers similar to that described above connected toa server. The connection between the computers and the server can bemade via a local area network (LAN), a wide area network (WAN), or viamodem connections, directly or through the Internet. The computernetwork allows information such as test scores, game statistics, andother subject information to flow from a subject's computer to a server.An administrator can review the information and can then downloadconfiguration and control information pertaining to a particular subjectto the subject's computer.

I. General Characteristics

FIG. 1 is a block diagram of one embodiment of a multi-faceted,web-deliverable, browser-playable and game-based neurological trainingsystem 1 configured to treat a cognitive deficit. The neurologicaltraining system 1 is described in more detail in U.S. patent applicationSer. No. 14/201,666 filed Mar. 7, 2014, entitled “NeuroplasticityGames,” which is herein incorporated by reference for all purposes.

The neurological training system 1 comprises a game platform 12 andtraining program 2. The game platform 12 controls exercise delivery andrecords all data on game play and performance progressions for exercisesuites playable on Internet-connected computers and pads. It comprises aprogram manager 3, participant portal 8, clinician portal 10, anddatabase 9. The program manager 3—also referred to herein as a gamemanager—is configured to administer the training program 2, manipulate aplurality of game stimuli 5, and receive input from at least one gamepiece 4.

The training program 2 comprises a plurality of games or exercises 4. Insome embodiments, the training program 2 targets a plurality ofcognitive domains and sensory modalities, from foundational ones likeprocessing speed to more complex ones like social cognition. Eachtraining program 2 is customized and configured to address cognitivedeficits that are associated with a neurological condition, such ashearing loss, addiction, depression, ADHD, or ASD, and itsco-morbidities.

In most embodiments, the game stimuli comprise images displayed on adisplay device such as a computer monitor or digital screen and/orsounds played through a speaker, ear buds or other auditory equipment.In other embodiments, the game stimuli comprise smells, tastes, ortactile (e.g., haptic) stimulation. The training program's stimulus setis designed to span the relevant dimensions of real-world stimuli toensure that learning is never stimulus specific.

Early in training, the games use highly salient, emphasized (e.g., highcontrast, temporally deliberate) stimuli to drive strongly synchronizedbrain responses requisite for rapidly driving brain changes in acorrective way. The games then progressively move to moreecologically-relevant and valid stimuli (e.g., real speech, complexrealistic social stimuli with people showing emotions in context, socialscenes, social interactions) to ensure generalization to real-worldsituations.

The game piece 6 comprises a keyboard, computer mouse, track pad, touchscreen, camera, remote sensing device (e.g., Microsoft Kinect®),microphone, or other input device.

The training program 2 provides the games through a portal 8 that isdesigned to be played in a social network environment, at a treatmentcenter, or during a treatment class. In one embodiment, the trainingprogram 2 is designed to be platform-independent so that it can bedelivered over the Web via any Internet-connected computer. In anotherembodiment, the training program 2 is provided through a hand-heldcomputer (iPhone/Android phone/iPad/Android tablet/Amazon Fire)application.

The participant portal 8 provides access to game participants.Practically any patient on any computer located anywhere in the worldcan work on these programs as frequently as their time and schedulepermit, under the supervision of a clinician who can (hypothetically)also be located anywhere. To use the program, a participant opens astandard web browser on a broadband connected computer and goes to aprogram web site. The participant then logs into the program using ascreen name that contains no personally identifiable information.

In one embodiment, the portal 8 introduces the participant to a“meta-game wrapper” such as an image and map of a virtual social citythat allows participants to visit locations, access games, view progressand results, and make or communicate with friends. The meta-game wrapperis characterized by simplicity, appealing graphics, a sense of control,and constant game rewards.

The program manager 7 administers a schedule that ensures that aparticipant progresses through the games 4 in a defined order, generallymoving from more simple (early sensory processing) games 4 to morecomplex (multimodal, working memory, memory span) games 4 over thecourse of a multi-week experience. At any point in time, the participanthas access to a subset (for example, eight) of these games 4, and ischallenged to perform at least a certain number (for example, six) ofthe games 4 per day. Each game 4 has specific criteria for completion orplateaued performance. After those criteria are met, the game 4 isremoved from the active set and the next game 4 added to it. Thismechanism ensures ongoing novelty and engagement for the participant,while ensuring that the participant progresses smoothly through thecomplete set over the program use period.

Within each game 4, a performance threshold, determined by anappropriate up-down procedure or a Zest algorithm, is derived for eachblock completed. The performance threshold provides performance andimprovement data on the individual games. Game-based assessments, whichare designated blocks with medium difficulty of the specific games 4,are performed at various time points in the intervention to checkprogress.

The games 4 in the training program 2 are designed with a differentobjective than conventional games. Conventional games start at a fixedbaseline and progress in a single direction, getting more difficultuntil the participant is unable to go any further, at which point thegame typically terminates. Conventional multilevel games also requirecompletion of one level to progress to the next, more difficult, level,terminating mid-level if the participant is unable to complete a givenlevel.

The games 4 of the training program 2, by contrast, are adaptive andthreshold-based. Learning rules are relaxed in initial training toassure near-errorless learning. Error rates are slowly increased toachieve challenge conditions known to be optimal for normal individualsby the end of the training epoch. Likewise, rewards or incentives forperformance gains are initially amplified, in comparison with thoseapplied for training in other patient populations. The games 4 increasein difficulty when the participant exceeds a threshold of success, andthey decrease in difficulty when the participant's performance dropsbelow another threshold. Many of the games 4 enable a participant to“unlock” a new level merely by beating the participant's previous bestscore. By measuring success in metrics of personal improvement ratherthan fixed performance requirements, both participants with relativelyhigh cognitive abilities and participants with relatively significantcognitive deficits can progress through the entire training program 2.

After logging in, a game-like experience begins in which the participantis encouraged to earn points and in-game rewards to further advance ineach game 4. To do so, the participant selects one of the games 4scheduled for the day, and performs that game for 5-10 minutes. The game4 itself contains the core science stimuli and task built into agame-like experience. Performing the game 4 resembles practice on askill akin to learning to play an instrument or learning to ride abicycle.

Participants perform tens to hundreds of trials over the course of theten-minute session. Each trial provides auditory and visual feedback andrewards to indicate if the trial was performed correctly or incorrectly.After each trial, the difficulty of the next trial is updated to ensurethat within each session, the participant gets ^(˜)85% of trialscorrect. Maintaining a relatively high level of success helps preventfrustration and minimizes the possibility of potential drop-out from theprogram. Summary screens including game metrics (points, levels) andgame metrics (usage, progress) are shown to the participant at the endof each session.

To progress through a game 4, the participant performs increasinglydifficult discrimination, recognition, memorization or sequencing tasksunder conditions of assured focused attention. Each game 4 employsadaptive tracking methods to continuously adjust one or two adaptivedimensions of the task to the sensory and cognitive capabilities of theparticipant. This process is based on a statistically optimal Bayesianapproach that allows the games 4 to rapidly adjust to an individual'sperformance level, and maintain the difficulty of the stimulus sets atan optimal level for driving most-efficient learning.

This continuously-adjusted adaptivity operates from trial to trial, tosustain an individual's performance success at a challenging (80-90%),since subject is not correct all the time, yet engaging and rewarding(since subject is correct most of the time) level of performancesuccess. This continuously-adjusted adaptivity is also adjusted acrosssessions to ensure that the games 4 become more challenging at exactlythe appropriate rate for a specific individual's rate of learning. Thisadaptivity also allows the game 4 to adapt to an individual's variableperformance across days depending on their overall mood, attention, andhealth.

By this strategy, training is individualized. A trainee rapidlyprogresses across training landscapes for which impairments are mild orinsignificant but must work hard to improve domains of substantialimpairment—always working on the edge of their achievable performanceabilities to drive positive, corrective changes at an optimizedday-by-day rate, to address the specific neurological problems that mostspecifically frustrate their higher functioning.

If a game 4 is used as a training module, it is presented as stages thatlast about ten minutes. During those ten minutes, the participant playsthe stage two times: first to set a baseline, and second to beat ormatch that baseline. This repetition is intentional, because thetargeted strengthening of certain neural pathways achieved in the gamesrequires repeated activation of those neural pathways. Stages generallyhave one level (block of trials intended to be played straight through),but they can have more.

The program manager 7 delivers all patient performance data in encryptedforms to a cloud-located database 9, which are provided, withappropriate informed consents, to one or more treatment programprofessionals, who access the relevant patient data through a clinicianportal 10 in order to supervise patient treatment and assure enrollment,compliance, and monitored and guided patient progress.

Every aspect of a patient's compliance and progress is recorded intraining and can be provided via the cloud-based database 9 (withappropriate permissions) to supervising training monitors orprofessionals. No personally identifiable information (includingInternet protocol addresses) is stored on the server. The server makesthe data available for review by the clinician(s) supervising patientcare, the site trainer, site coordinator, and site investigator througha secure web portal 10, which requires a complex password to secure theidentification of these individuals. Only data from participants in aparticular clinic can be viewed by that clinic's staff. The sitetrainer, in particular, uses the secure web portal 10 to regularly checkon usage and progress of each active participant to customize theirweekly phone/in-person/social network discussions to provide helpfulguidance and coaching.

II. Assessments

Each training program 2 utilizes assessments to personalize the types ofgames, types of stimuli, and levels of difficulty to the participant.Each game 4 can be used as an assessment or a training module. If a game4 is used as an assessment, it is played once through before theparticipant advances to the next game.

Playing an assessment typically lasts five or fewer minutes. Assessmentsare used sparingly to avoid inducing learning/practice effects. Inassessments, progressive variables that change an exercise's difficultymay be removed to provide a static reading of performance. Visual and/orauditory feedback and rewards may also be removed to reduce effectsrelated to trial-by-trial learning or motivation.

Assessments are embedded in the training program 2 in form of surveysand exercises of varying similarity to those selected for training.Assessments are placed before and after training and often duringcheckpoints within training. Checkpoints generally occur every 10 to 14days of training and can be spaced further apart for longer trainingprograms.

The training program 2 calibrates the games 4 based on pre-trainingassessment results to select games 4 and certain stages or levels withinthe games to present as part of the training program. Theassessment-based calibration means adjusting the type and salience ofstimuli. It some embodiments, the calibration also means adjusting theproportion of training exercises related to each cognitive domain (e.g.,processing speed, attention, theory of mind, impulse control).Calibration is also used to adjust parameters within an exercise likethe start point of the progressive variable or the number of trials in ablock. Such parameters tend to be calibrated for adults with moderatelyhigh cognitive abilities. Young children and individuals suffering fromcognitive impairments often require specialized settings.

1. Hearing Profile Assessment

FIGS. 2-4 illustrate screenshots 20, 22 and 24 of one embodiment of ahearing profile assessment that measures a participant's hearing loss.Hearing thresholds are obtained from 0.25-16 kHz, spanning the speechfrequency range. Hearing thresholds for each frequency are obtainedindependently by finding the quietest volume level the frequency can beplayed at that the participant consistently judges to be audible. Thehearing profile assessment is referred to herein as themultiple-interval staircase (MIS) assessment.

In FIG. 2, the assessment prompts the participant to maintain aconsistent, quiet and controlled environment each time they take thehearing test to ensure conditions are comparable across tests. In FIG.3, the assessment prompts the participant to calibrate their computervolume to a comfortable level to ensure enough test sounds will beaudible yet not painful. In FIG. 4, the assessment presents images ofmultiple birds 26 on a screen. Each bird plays the same test frequencyat different controlled loudness levels. The assessment instructs theparticipant to hover the mouse over each bird 26 and click it if theycan hear it. If the participant hovers their cursor for too long overany single bird 26, the bird disappears and that decibel level for thatfrequency is considered to be inaudible. This process is repeated atquieter loudness levels until the participant consistently chooses thesame loudness (decibel) level as the quietest bird they indicated asaudible. This maximizes response consistency and minimizes the influenceof uncontrolled environmental sounds.

FIG. 5 is a functional block diagram of one embodiment of the MISaudiometry exercise and assessment 50. In block 51, the assessment 50plays each of a plurality of tones or sound samples multiple times at aplurality of loudness levels, such as 10 dB loudness increments. Withinblock 51, each tone or sound sample is presented at a common frequency.Only the loudness level differs.

Block 52 illustrates a screenshot showing how the assessment 50 is, inthis exemplary embodiment, embedded in a game that displays a pluralityof visual objects over a scenic background. In this particular example,nine birds are shown, and each bird represents a different loudnesslevel or a silent foil. As shown in block 52, the assessment 50challenges the participant to move a mouse pointer over every bird andselect the ones the participant can hear. The assessment 50 alsochallenges the participant to respond as fast as possible, illustratinga time bar that gives the participant a limited amount of time to selectthe bird. The assessment 50 limits the number of times a participant canhover the pointer over a bird to listen to the stimulus associated withthat bird.

After detecting—based on a combination of mouse movements and possiblemouse selections—that the user is done listening to all, or at least asufficient number, of the tones, then flow proceeds to block 53. If theuser indicated that any of the tones were audible, and the user'scumulative selections do not suggest an inordinate amount of errorexhibited by foil selection or a distribution of selections that are notsufficiently positively correlated with loudness, then flow proceeds toblock 57. If the user did not indicate that any of the tones wereaudible, then flow proceeds to blocks 54, 55, and 56. If the user'scumulative selections suggest an inordinate amount of error, then theassessment 50 repeats block 52 using the same loudness levels or thesame median loudness level but with a greater distribution of loudnesslevels. Alternatively, the assessment 50 terminates.

Block 54 illustrates a scenic screenshot of birds flying in darkness,symbolizing the participant's inability to hear. After displaying thisscreenshot, the assessment 50 in block 55 increases the amplitude ofeach tone or sound sample by 30 dB and, in block 56, retests the tonesor sound samples at the same frequency most recently tested.

Block 57, by contrast, illustrates a scenic screenshot of birds flyingin daylight, symbolizing the participant's ability to hear. In block 58,the assessment 50 identifies the threshold loudness level, which iseither the quietest selected loudness level or a statistical loudnesslevel conservatively representative of the lower threshold of theparticipant's hearing.

Flow proceeds to block 59. If the last time the assessment 51 performedthe actions illustrated with respect to block 52 was with the samefrequency, and the identified threshold loudness level is consistentwith—that is, either the same as or within a range of—the lastidentified threshold loudness level for that same frequency, then flowproceeds to block 61. Otherwise, flow proceeds to blocks 60 and 56, andthe assessment 50 tests the same frequency again, but using loudnesslevels that are distributed more narrowly (e.g., 4 dB increments) aroundthe identified threshold loudness level.

Once a consistent threshold loudness level is determined, then in block61 the detected threshold loudness level is treated as the assessedthreshold loudness level for that frequency. Flow proceeds to block 62,in which the assessment 50 tests the participant at a new frequency,repeating the actions illustrated at blocks 51 and 52. In this manner,the assessment 50 adapts the loudness levels of each frequency toneuntil a consistent minimum-amplitude threshold is identified.

Once the assessment 50 is completed, the training program 2 selectsstimuli to use in a plurality of neuroplasticity games 4 that are afunction of the results obtained from the audiometric exercise. Some ofthe neuroplasticity games 4 are auditory perceptual training games thatuse game stimuli comprising auditory signals.

2. Tinnitus Assessment

FIGS. 6-10 illustrate screenshots 70, 75, 77, 89 and 91 of oneembodiment of a tinnitus assessment. The assessment measures the extentto which tinnitus disrupts the participant's life. In FIG. 6, theassessment presents a questionnaire asking the participant to rate theimpact of their tinnitus on their well-being. Questions are from theclinically-applied rating systems Tinnitus Handicap Inventory andTinnitus Functional Index. In FIG. 7, the assessment encourages theparticipant to use headphones to help control sound quality and ensurethe sounds are presented clearly and consistently. In FIG. 8, theassessment prompts the participant to select a sound that best matchesthe sound of their subjective tinnitus. The participant changes one ormore sound parameters, such as frequency, by moving the cursor across anarea on the screen, and then selecting when the closest matching soundis found. In FIG. 9, the assessment prompts the participant to indicatehow well the sound they chose matches what their tinnitus actuallysounds like. In FIG. 10, the assessment shows an image of a head andprompts the participant to select the place on the head where thetinnitus sounds like it comes from.

III. Interoaudioceptive Calibration Procedure

Most people have observed that the quality of one's own voice whenspeaking sounds different from one's voice replayed on from soundrecording device. When a person perceives their own speech production as“sounding perfectly correct,” they are referring to a more resonant,lowered pitch version of their voice received through the combination ofconduction through the tissues of the head and conduction through theair.

A person perceives their own voice through two sources of stimulus—theexternal stimulus caused by the sound/pressure waves leaving one's mouthand reaching their ear, and the internal stimulus caused by vibrationstranslated through the neck and skull. While other people receive onlythis external stimulus, a person speaking receives both the internal andexternal stimulus.

This asymmetry is profoundly exemplified by many patients with aidedhearing or speech. Such patients often describe the speech that theyproduce as “completely normal” and “completely understandable,” evenwhile they find the speech of others received by a cochlear implant orhearing aid to be completely or largely unintelligible.

We believe that this asymmetry can be leveraged to improve the degradedauditory processing abilities of a hearing-impaired individual. Inparticular, we believe that recovery or improvements in speechunderstanding can grow from the gradual neurological establishment ofcorrespondences between the patient's own voice model and the voices ofother speakers.

FIGS. 11-13 illustrate three embodiments of interoaudioceptivecalibration procedures and systems.

FIG. 11 illustrates an embodiment of an interoaudioceptive calibrationprocedure and system 100 for adjusting the sound processing parametersof a hearing aid or cochlear implant. In block 101, the system 100instructs the subject, if possessing or being fitted with a hearing aid,to wear the hearing aid. This block can be skipped in the case of a userwearing a cochlear implant.

In block 102, the system 100 instructs the subject to vocalize a sound(such as a word or a tone) and remember the sound that they vocalize. Inblock 103, the system 100 records the subject's voice as they vocalizethe designated sound. In block 104, the system 100 plays the recordedvoice back to the subject.

In block 105, the system 100 queries the subject whether the recordingthey heard matches, or sounds similar to, their own memory of theirvoice. In an alternative embodiment, the system 100 queries the subjectto identify which of two or more recordings sounds the best or bestmatches the user's memory of their voice. In yet another alternativeembodiment, the system 100 queries the subject whether the pitch of therecording they heard sounds higher, lower, or approximately the same asthe user's memory of their voice. The system 100 may also oralternatively query the subject whether the timbre, resonance, frequencyat the high, middle, or low end of the spectrum, or other sound qualitymatches the user's memory of their voice.

If the subject indicates that the sounds are dissimilar (or a moreoptimal match), then flow proceeds to block 106. Otherwise, flowproceeds to block 107.

In block 106, the system 100 adjusts one or more sound processingparameters. In one embodiment, the sound processing parameters are theparameters used by the hearing aid or cochlear implant to process sound.In another embodiment, the sound processing parameters are parametersused to adjust the sound of the recorded voice as it is played back.

Based on the user's feedback, the system 100 identifies differencesbetween the user's mixed intero- and extero-audioception of their ownvoice and the user's exteroaudioception of their own voice. The system100 effectively reduces these differences to a sound-processingalgorithm, using a progressive matching strategy that alters a recordedspeech replay of a patient's voice to match the qualities of one's ownvoice heard while speaking. The system 100 repeatedly plays recordedsamples of the patient's voice, procures feedback from the patientregarding their comparisons of the recorded samples with the sound orsounds they remembered vocalizing, and adjusts the sound processingparameters until the patient indicates that they are satisfied that therecorded samples sound similar to what they remembered vocalizing, athreshold number of adjustments have been made, or an evaluation of thepatient's feedback indicates a marginal benefit to further adjustments.

Then, in block 107, the program 100 uses the algorithm describing thealtered speech to adjust the cochlear implant or hearing aid soundprocessing parameters to shape the hearing of all other voices to matchthe characteristics of the patient's voice received while speaking.

Finally, in step 108, the system 100 optionally gradually renormalizesthe sound processing parameters over a period of time or course oftraining, reducing the distortion required to make hearingunderstandable and rendering a patient's hearing, with the aid of thehearing aid or cochlear implant, more realistic. In this manner, afterthe subject masters speech reception with this initial cochlear implantor hearing aid parameter setup, the distortions in speech used to matchthe speech of others with the self-received speech of the patient areprogressively lessened in a series of automatic or audiologist or otherclinician-implemented steps until, over a variable period of weeks ormonths, all speech is heard in its more variable, natural forms.

A more elaborate implementation of the interoaudioceptive calibrationprocedure and system 100 also instructs the patient to produce soundsamples, and records the same, that exhibit functionally significantmodulation characteristics such as pace, depth of modulation, rhythm,and other prosodic features. Moreover, as a subject masters his or herlistening tasks, the algorithmic distortion related to that modulationcharacteristic (spectral and temporal and timbre and rhythm and in otherways prosodically matched voice) is progressively reduced in steps onthe path of normal received speech.

FIG. 12 illustrates an embodiment of an interoaudioceptive calibrationprocedure and system 130 for improving the efficacy of auditory andcognitive training exercises. In block 131, the system 130 instructs thesubject to vocalize a sound (such as a word or a tone) and remember thesound that they vocalize. In block 132, the system 130 records thesubject's voice as they vocalize the designated sound. In block 133, thesystem 130 alters the recorded voice using sound processing parameters.In block 134, the system 130 plays the recorded voice, altered via thesound processing parameters, back to the subject.

In block 135, the system 130 queries the subject whether the recordingthey heard matches, or sounds similar to, their own memory of theirvoice. In an alternative embodiment, the system 130 queries the subjectto identify which of two or more recordings sounds the best or bestmatches the user's memory of their voice. In yet another alternativeembodiment, the system 130 queries the subject whether the pitch of therecording they heard sounds higher, lower, or approximately the same asthe user's memory of their voice. The system 130 may also oralternatively query the subject whether the timbre, resonance, frequencyat the high, middle, or low end of the spectrum, or other sound qualitymatches the user's memory of their voice.

If the subject indicates that the sounds are dissimilar (or a moreoptimal match), then flow proceeds to block 136. Otherwise, flowproceeds to block 137.

In block 136, the system 130 adjusts one or more sound processingparameters used, in step 133, to play the user's recorded voice. Flowproceeds back to step 131.

Based on the user's feedback, the system 130 identifies differencesbetween the user's mixed intero- and extero-audioception of their ownvoice and the user's exteroaudioception of their own voice. The system130 effectively reduces these differences to a sound-processingalgorithm, using a progressive matching strategy that alters a recordedspeech replay of a patient's voice to match the qualities of one's ownvoice heard while speaking. Then, in block 137, the program 130 uses thealgorithm describing the altered speech to select the auditory stimuli,and parameters (such as pitch and volume) of the auditory stimuli,played in auditory and cognitive training games and exercises.

Finally, in step 138, the system 130 optionally gradually renormalizesthe sound processing parameters over a period of time or course oftraining, reducing the distortion required to make hearingunderstandable and rendering a patient's hearing more realistic.

A more elaborate implementation of the interoaudioceptive calibrationprocedure and system 130 also instructs the patient to produce soundsamples, and records the same, that exhibit functionally significantmodulation characteristics such as pace, depth of modulation, rhythm,and other prosodic features. Moreover, as a subject masters his or herlistening tasks, the algorithmic distortion related to that modulationcharacteristic (spectral and temporal and timbre and rhythm and in otherways prosodically matched voice) is progressively reduced in steps onthe path of normal received speech.

FIG. 13 illustrates yet another embodiment of an interoaudioceptivecalibration procedure and system 150. In block 151, the system 150instructs the subject to vocalize a sound (such as a word or a tone) andremember the sound that they vocalize. In block 154, the system 150plays back one or more synthesized speech items, or recorded speech ofother people, using one or more sound processing parameters. The speechitems can, for example, be vowels or phonemic sounds, or syllables,words, phrases, or connected speech. Using initial speech items that arelike the speaker's heard self-produced speech at the initial stage oftraining makes it easier for the subject to recognize thealgorithmically produced voices of others.

In block 155, the system 150 queries the subject whether the synthesizedspeech recording or other-voice recording they heard matches, or soundssimilar to, their own memory of their own voice. In an alternativeembodiment, the system 150 queries the subject to identify which of twoor more recordings sounds the best or best matches the user's memory oftheir voice. In yet another alternative embodiment, the system 150queries the subject whether the pitch of the recording they heard soundshigher, lower, or approximately the same as the user's memory of theirvoice. The system 150 may also or alternatively query the subjectwhether the timbre, resonance, frequency at the high, middle, or low endof the spectrum, or other sound quality matches the user's memory oftheir voice.

If the subject indicates that the sounds are dissimilar (or a moreoptimal match), then flow proceeds to block 156. Otherwise, flowproceeds to block 157.

In block 156, the system 150 adjusts one or more sound processingparameters used, in step 154, to play the synthesized speech items orexcerpts of other voices. Flow proceeds back to step 151.

Based on the user's feedback, the system 150 identifies differencesbetween the user's mixed intero- and extero-audioception of their ownvoice and the user's exteroaudioception of their own voice. The system150 effectively reduces these differences to a sound-processingalgorithm, using a progressive matching strategy that alters a recordedspeech replay of a patient's voice to match the qualities of one's ownvoice heard while speaking. Then, in block 157, the program 150 uses thealgorithm describing the altered speech to select the auditory stimuli,and parameters (such as pitch and volume) of the auditory stimuli,played in auditory and cognitive training games and exercises.

Finally, in step 158, the system 100 optionally gradually renormalizesthe sound processing parameters over a period of time or course oftraining, reducing the distortion required to make hearingunderstandable and rendering a patient's hearing more realistic.Moreover, as a subject masters his or her listening tasks, thealgorithmic distortion related to that modulation characteristic(spectral and temporal and timbre and rhythm and in other waysprosodically matched voice) is progressively reduced in steps on thepath of normal received speech.

A more elaborate implementation of the interoaudioceptive calibrationprocedure and system 130 also instructs the patient to produce soundsamples that exhibit functionally significant modulation characteristicssuch as pace, depth of modulation, rhythm, and other prosodic features.The system 130 also plays synthesized voice samples or actual voicesamples from others that exhibit the same modulation characteristics,and challenges the subject whether they match.

In an important variation of this implementation, training extends toaccurately receiving the voices of others heard at low volume (soundintensities), or against a background of environmental noises.

All of the aforementioned strategies accelerate the rate of acquisitionof speech understanding or speech reception improvement in speech andhearing assisted patients.

To summarize, it will be seen that the interoaudioceptive calibrationprocedures and systems improve the user's speech recognition and generalhearing by normalizing specific perceptual deficits that contribute totheir hearing loss. The interoaudioceptive calibration procedures andsystems may include regularly decoding the user's personal hearingprofile to regulate an optimal training schedule that may be used withor without a hearing-aid.

Some embodiments of the training program 2 include an internet-basedinteroaudioceptive calibration system and procedure for improvinghearing of hearing-impaired populations. The perceptual trainingconsists of auditory discrimination exercises that improve the user'sability to judge low-level sound features such as timing, frequency andintensity, as well as high-level structures such as phonemes. Theschedule involves training the user to initially discriminateexaggerated sounds and refining these distinctions to natural levels asthey become able to make more sensitive distinctions. The incorporationof interoaudioceptive calibration procedures accelerates and improvesthe quality of the reacquisition of improvement in speech reception inan individual who has received a cochlear implant or hearing aid byleveraging the speech received from a user's voice as a guide forre-acquiring new or improved speech understanding.

Those skilled in the art should appreciate that they can readily use thedisclosed conception and specific embodiments as a basis for designingor modifying other structures for carrying out the same purposes of thepresent invention without departing from the spirit and scope of theinvention as defined by the appended claims. For example, variousembodiments of the methods disclosed herein may be implemented byprogram instructions stored on a memory medium, or a plurality of memorymedia.

We claim:
 1. An audiometry method comprising: providing a computer gameenvironment for an audiometric exercise, the game environment displayinga plurality of visual stimuli that represent objects in a naturalenvironment or video game theme, wherein each of a first subset of thevisual stimuli are associated with an audio stimulus, and each of asecond subset of the visual stimuli is not associated with an audiostimulus, and a means is provided of selecting any of the plurality ofvisual stimuli; wherein each audio stimulus comprises a fixed frequencytone associated with a fixed loudness level, and the loudness levels ofeach audio stimulus is separated from each other audio stimulus bydiscrete decibel increments; and performing the audiometric exercise by:(a) displaying the plurality of visual icons; (b) for each of at leasttwo or more of the plurality of visual icons: receiving an initialselection of the visual icon to play audio stimulus associated with thevisual icon; playing the audio stimulus, if any, associated with thevisual icon; and if the user hears the audio stimulus associated withthe visual icon, receiving the user's second selection of the visualicon to indicate that the user can hear the audio stimulus; (c) when athreshold loudness level is identifiable, identifying a thresholdloudness level for the frequency tone of the audio stimuli and reducinga decibel increment between the loudness levels associated with thevisual icons; and (d) when no threshold loudness level is identifiablebecause the user does not make any second selections, increasing theloudness levels associated with the plurality of visual icons.
 2. Theaudiometry method of claim 1, further comprising challenging the user,in the computer game environment, to indicate as rapidly as possiblewhether they can hear the frequency tones, and providing sensoryfeedback on an accuracy and speed of the user's response.
 3. Theaudiometry method of claim 2, further comprising: displaying to theuser, in the computer game environment, a plurality of visual objects,each associated with one of the tones; and challenging the user to hovera pointer over the objects to play the tones and to select the objectsif they are able to hear the tones.
 4. The audiometry method of claim 1,further comprising: selecting stimuli to use in a subsequent game thatare a function of results obtained from the audiometry exercise.
 5. Theaudiometry method of claim 4, wherein the subsequent game is an auditoryperceptual training game, and the game stimuli comprise auditorysignals.
 6. The audiometry method of claim 1, further comprising foreach frequency tone: identifying a minimum-amplitude threshold for thefrequency tone as the quietest loudness level that the user indicatedthat they could hear; subsequent to identifying the minimum-amplitudethreshold, playing the frequency tone at a plurality of loudness levelsdistributed around the minimum-amplitude threshold; and repeating thepreceding two steps until the user repeatedly selects loudness levelsfor the frequency tone that are within a threshold of each other.
 7. Theaudiometry method of claim 6, wherein the plurality of loudness levelsat which the frequency tone is played are uniformly distributed acrossdiscrete decibel increments.
 8. The audiometry method of claim 7,further comprising: after determining a minimum-amplitude threshold froma first distribution of loudness levels, playing the frequency tone at asecond distribution of loudness levels having smaller discrete decibelincrements.
 9. The audiometry method of claim 5, wherein for eachfrequency tone, if the user does not indicate hearing at any of a firstplurality of loudness levels, playing the frequency tone again at asecond plurality of loudness levels that are distributed about a meanloudness level that is louder than a mean loudness level of the firstplurality of loudness levels.
 10. The audiometry method of claim 5,further comprising: after identifying a consistent minimum-amplitudethreshold for a first frequency, playing a tone at another frequencymultiple times at a plurality of loudness levels until a consistentminimum-amplitude threshold is identified; and repeating the precedingsteps until a consistent minimum-amplitude threshold is identified foreach frequency in the distribution of frequencies, wherein consistentmeans within a threshold error range.
 11. The audiometry method of claim1, wherein the means of selecting any of the plurality of stimulicomprises a keyboard, mouse, trackpad, touch screen, microphone, camera,or other sensor.